Scaling Down EM Using Graph Based Models

The EM algorithm is used extensively in data mining problems that involve uncertainty in the form of missing values or latent variables. It was recently reported in KAIS as being one of the top 10 data mining algorithms. Typical applications include indoor map building, mixture models and image mining. However, as data mining applications extend to deployment beyond traditional desk top machines using typical probabilistic parametric models is infeasible. Resource constrained computational platforms such as hand-held GPS systems, small WiFi devices, and Sony Aibo robots have slow processors and no or limited floating point routines and hence are difficult to implement probabilistic formulations of EM on. In this paper we investigate the EM algorithm with undirected graphs as models. Constraints on the graph topology lead to different model classes and we explore several we believe have practical applications. We prove that the E-step reduces to the st-mincut problem which can be solved in polynomial time. For two of our model classes, we derive a non-heuristic M-step that can be carried out in polynomial time. These algorithms do not require floating point hardware and we demonstrate their performance on several classic problems.